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Abstract 

To  ensure  aviation  safety,  air  traffic  safety  agen¬ 
cies  have  required  minimum  aircraft  separation  stan¬ 
dards.  Currently,  the  FAA  requires  a  minimum  sep¬ 
aration  of  3  nautical  miles  (NM)  when  aircraft  are 
within  40  NM  of  an  air  traffic  control  surveillance 
radar  and  a  separation  of  5  NM  when  aircraft  are 
located  beyond  40  NM  from  the  radar.  At  the  same 
time,  the  FAA  permits  a  separation  of  3  NM  out 
to  60  NM  for  single  sensor  terminal  systems  using 
an  ASR-9  Mode  S  MSSR.  The  ASR-9  Mode  S  and 
ASR-11  Monopulse  Secondary  Surveillance  Radar 
(MSSR)  are  similar  monopulse  systems  and  the  40 
NM  limit  on  terminal  separation  minima  for  the  ASR- 
11  may  be  unnecessarily  restrictive.  In  this  paper, 
we  seek  to  determine  if  there  are  any  differences  in 
radar  azimuth  performance  between  the  ASR-9  Mode 
S  and  the  ASR-11  at  ranges  of  40  to  60  NM.  To 
perform  this  analysis  we  implement  a  method  that 
estimates  radar  error  from  raw  time-stamped  range 
and  azimuth  reports  of  aircraft  flying  through  the 
airspace  (targets  of  opportunity).  The  method  filters 
the  data  to  extract  radar  reports  from  aircraft  traveling 
at  nearly  constant  heading,  velocity  and  altitude,  and 
then  utilizes  knowledge  of  the  aircraft  dynamics  to 
accurately  estimate  the  true  aircraft  position.  Our 
analysis  employs  a  reference  system  approach  com¬ 
paring  the  performance  of  the  alternative  system,  the 
ASR-11,  against  the  performance  of  the  approved 
legacy  system,  the  ASR-9  Mode  S,  to  determine  if 
the  alternative  system’s  performance  is  equivalent  or 
better. 

Introduction 

Current  FAA  regulation,  Order  JA  7110.65T  5- 
5-4  [1],  states  that  when  using  an  ASR-11  with 
Monopulse  Secondary  Surveillance  Radar,  aircraft 
must  be  separated  by  a  minimum  of  3  nautical  miles 
(NM)  when  less  than  40  NM  from  the  radar  antenna 


and  a  minimum  of  5  NM  when  40  NM  or  more  from 
the  radar  antenna.  The  same  regulation  extends  the 
permitted  use  of  3  NM  separation  minima  out  to  60 
NM  for  single  sensor  terminal  systems  using  an  ASR- 
9  with  Mode  S.  The  ASR-1 1  and  the  ASR-9  Mode  S 
are  similar  monopulse  systems,  therefore  the  40  NM 
limit  on  terminal  separation  minima  for  the  ASR- 
11  may  be  unnecessarily  restrictive.  Consequently, 
the  FAA  Flight  Systems  Laboratory,  AFS-450,  com¬ 
missioned  a  study  to  evaluate  ASR-11  performance 
and  determine  if  the  ASR-11  can  support  3  NM 
separation  minima  at  ranges  of  greater  than  40  NM. 
As  part  of  this  study,  MIT  Lincoln  Laboratory  was 
tasked  with  completing  a  large  scale  data  analysis 
of  ASR-11  performance.  The  results  from  this  effort 
are  presented  in  this  paper.  The  AFS-450  study  also 
includes  three  additional  components  which  include 
evaluation  of  controlled  flight  test  data,  factory  testing 
of  the  ASR-11,  and  an  analytical  comparison  of  the 
radar  systems.  The  MIT  Lincoln  Laboratory  effort 
will  be  considered  in  conjunction  with  these  other 
components  when  determining  the  suitablility  of  the 
ASR-11  for  extended  3  NM  separation  and  does  not 
on  its  own  provide  sufficient  breadth  to  support  a 
change  in  operational  procedure. 

The  analysis  evaluates  the  suitability  of  the  ASR- 
11  for  supporting  3  NM  separation.  A  reference 
system  approach  is  employed,  comparing  the  perfor¬ 
mance  of  the  alternative  system,  ASR-11,  against  the 
performance  of  the  approved  legacy  system,  the  ASR- 
9  Mode  S,  to  determine  if  the  ASR- 1 1  performance 
is  equivalent  or  better.  The  radar  errors  are  calculated 
using  a  technique  developed  in  [2]  to  estimate  radar 
azimuth  error  from  large,  widely  distributed  sets  of 
radar  data.  Additional  validation  of  the  technique  is 
performed  to  verify  that  is  not  dependent  on  range 
or  elevation  angle;  two  factors  explored  in  this  paper 
for  their  affect  on  radar  azimuth  error.  Over  4  million 
error  samples  are  collected  from  each  radar  type  and 
analyzed.  The  error  characteristics  for  each  radar  type 
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are  then  calculated  and  broken  down  by  range,  eleva¬ 
tion  and  altitude  for  a  comprehensive  comparison  of 
the  systems.  Finally,  conclusions  on  the  performance 
differences  between  the  ASR-11  and  ASR-9  Mode  S 
are  presented. 

Background 

Monopulse  Secondary  Surveillance  Radar 

The  radar  systems  discussed  in  this  paper  consist 
of  primary  radar  with  co  located  beacon  interrogators, 
referred  to  as  Monopulse  Secondary  Surveillance 
Radar  (MSSR).  Beacon  interrogators  are  not  techni¬ 
cally  radar  but  utlize  radar-like  technology  [3].  Unlike 
primary  radar,  which  detect  targets  from  the  reflected 
energy  of  a  transmit  pulse,  beacon  interrogators  de¬ 
tect  compliant  aircraft  by  sending  an  interrogation 
signal  to  a  transponder  on  board  the  aircraft  and 
then  processing  the  elicited  response.  Beacon  systems 
are  often  referred  to  as  "one-way"  as  they  operate 
on  transmissions  travelling  from  the  target  to  the 
radar  antenna.  This  is  in  contrast  to  radar  which  rely 
on  a  "two-way"  signal  travelling  from  the  radar  to 
the  target  and  then  reflecting  back  to  the  radar.  The 
monopulse  in  MSSR  refers  to  the  technique  used  to 
measure  the  azimuth  of  the  target.  Monopulse  systems 
are  more  accurate  and  require  fewer  interrogations 
per  aircraft  than  traditional  "sliding  window"  systems, 
and  consequently  are  the  preferred  and  most  widely 
used  system  in  the  National  Airspace  System  (NAS) 

[4]. 

This  analysis  evaluates  the  performance  of  the 
beacon  systems,  and  does  not  consider  the  perfor¬ 
mance  of  the  primary  radar.  In  this  paper  the  terms 
ASR-11  and  ASR-9  refer  specifically  and  exclusively 
to  the  MSSR  component  of  the  radar  system.  Addi¬ 
tionally  in  the  case  of  the  ASR-9,  the  term  ASR-9 
refers  specifically  and  exclusively  to  an  ASR-9  with 
a  co- located  Mode  S. 

Radar  Azimuth  Errors 

Radar  azimuth  errors  consist  of  a  combination  of 
two  different  errors:  azimuth  measurement  error  and 
radar  azimuth  bias.  Azimuth  measurement  error  is  the 
error  in  the  measurement  of  the  aircraft’s  azimuthal 
position  relative  to  the  radar  beam.  Radar  azimuth 
bias  is  a  systematic  error  caused  by  a  misalignment 
of  the  radar  with  true  north.  In  the  existing  single 
sensor  radar  system,  radar  azimuth  bias  has  minimal 


effect  on  surveillance  performance  with  respect  to 
separation  services.  Single  sensor  radar  systems  apply 
a  sensor’s  azimuth  bias  equally  to  all  aircraft  and 
the  bias  is  in  effect  cancelled  out  in  any  separation 
measurements.  All  radar  errors  mentioned  in  this 
paper  refer  solely  to  the  azimuth  measurement  error 
and  do  not  account  for  any  radar  azimuth  bias. 


Azimuth  Error  Estimation  Technique 

The  radar  error  estimation  technique  developed 
in  [2]  accurately  estimates  radar  azimuth  error  using 
raw  time-stamped  range/azimuth  radar  reports  from 
aircraft  flying  through  the  NAS,  known  as  targets 
of  opportunity  (TOO),  which  provides  a  method  for 
analyzing  large  quantities  of  radar  data.  The  method 
filters  TOO  data  so  that  it  contains  only  aircraft  flying 
straight  and  level  at  a  near  constant  velocity,  and  then 
uses  this  a  priori  knowledge  of  the  aircraft  behavior 
to  accurately  estimate  the  true  aircraft  position  and 
radar  error. 

The  estimation  technique  involves  a  multi-step 
process.  First,  tracks  of  raw  secondary  reports  from  a 
single  sensor  recorded  in  radar  coordinates  (azimuth, 
slant  range,  and  altitude)  are  projected  onto  a  stere¬ 
ographic  plane  and  stored  in  Cartesian  coordinates 
(x,y)  relative  to  the  sensor.  Each  track  is  then  passed 
through  a  filter  that  calculates  the  smoothed  heading 
and  velocity  of  the  aircraft  and  extracts  periods  of 
straight  and  level  flight  at  a  near  constant  velocity. 
The  straight  and  level  tracks  are  then  fed  to  the 
estimation  algorithm.  The  algorithm  estimates  the  true 
trajectory  of  the  aircraft  as  the  line  that  best  fits  the 
track  in  a  least-squares  sense.  The  algorithm  estimates 
the  true  position  of  the  aircraft  at  the  time  of  each 
radar  measurement  by  finding  the  set  of  points  {xi,yi) 
on  the  least-squares  line  that  minimize  the  summed 
distance  between  the  points  and  the  measurements 
(xj,yi),  under  the  constraint  of  the  aircraft  flying  at 
a  constant  ground  speed  v.  More  precisely,  we  are  in 
search  of  the  following  set: 
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where  di  is  the  distance  between  the  points  (xt,yi)  and 
(■*«)?/)>  and  ri+1and  tf  are  the  time  stamps  of  consec¬ 
utive  radar  reports.  The  estimated  true  positions  are 
then  converted  back  into  radar  coordinates  and  the 
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Figure  1.  Estimation  method  validation 


azimuth  error  is  calculated  for  each  measurement.  A 
detailed  explanation  and  validation  of  this  estimation 
technique  is  found  in  [2]. 

Additional  Estimation  Method  Validation 

The  analysis  in  this  paper  explores  radar  azimuth 
error  dependence  on  range  and  elevation  angle.  To 
ensure  the  validity  of  the  results,  the  estimation 
method  accuracy  must  be  shown  to  be  independent  of 
those  factors.  To  supplement  the  general  performance 
evaluation  performed  in  [2],  an  analysis  of  the  estima¬ 
tion  method’s  accuracy  relative  to  range  and  elevation 
angle  was  performed.  The  analysis  generated  10,000 
random  aircraft  tracks  which  were  tracked  by  a  sim¬ 
ulated  ASR-9  radar  using  the  accepted  error  model 
for  ASR-9  Mode  S  azimuth  error,  which  consists 
of  a  zero  mean  normal  distribution  with  a  standard 
deviation  of  0.068°  [5].  The  simulated  radar  reports 
were  processed  by  the  estimation  method  and  the 
resulting  azimuth  errors  are  presented  in  Figure  1. 
The  estimation  accuracy  is  broken  down  by  report 
range  and  elevation  angle.  The  simulated  ASR-9  error 
model  [5]  does  not  include  any  range  or  elevation 
angle  dependent  errors,  so  the  simulated  error,  shown 
in  blue,  looks  very  similar  across  all  the  plots.  In 
each  sublot  of  Figure  1  the  estimated  azimuth  er¬ 


ror,  illustrated  as  a  red  line,  accurately  captures  the 
simulated  true  error,  shown  as  a  blue  histogram.  The 
estimation  method  performance  shows  no  dependence 
on  range  or  elevation  angle  and  can  therefore  be  used 
to  estimate  the  azimuth  error  dependence  on  those 
same  factors. 

Analysis 

Data  Collection 

Analysis  was  conducted  on  a  collection  of  radar 
data  supplied  by  the  84th  Radar  Evaluation  Squadron 
(RADES).  The  data  set  consisted  of  14  days  of  radar 
data  from  February  28th  to  March  13th,  2009.  Radar 
reinforced  beacon  reports  were  collected  from  a  total 
of  14  different  radar;  7  ASR-9  and  7  ASR-11.  The 
ASR-9  and  ASR-11  sensors  chosen  for  the  analysis 
are  listed  in  Tables  I  and  II  respectively. 

The  radar  reports  consisted  of  range,  azimuth 
and  pressure  altitude  measurements.  Prior  to  projec¬ 
tion  onto  a  stereographic  plane  the  pressure  altitude 
measurements  were  corrected  using  local  weather 
data.  Aircraft  altitude  is  reported  as  Mode  C  pressure 
altitude  from  sea  level  on  an  average  day,  baromet¬ 
ric  pressure  of  29.92  mm  Hg.  As  local  barometric 
pressure  deviates  from  this  standard.  Mode  C  alti¬ 
tude  reports  become  inaccurate.  To  ensure  accurate 
position  projections  the  Mode  C  altitude  reports  were 
corrected  to  the  true  altitude  using  the  local  hourly 
barometric  pressure  and  temperature  from  online 
sources  [6].  Each  altitude  corrected  radar  report  was 
then  projected  onto  a  stereographic  plane  tangential 
to  its  sensor  and  stored  in  a  database  in  both  radar  and 
Cartesian  coordinates.  The  local  weather  data  required 
for  Mode  C  altitude  correction  was  incomplete  on 
some  days.  Therefore  data  from  these  periods  were 
not  included  in  the  analysis.  Using  the  estimation 
technique  in  [2],  radar  azimuth  errors  were  calcu¬ 
lated  for  all  reports  from  ASR-11  sensors  and  for 
all  monopulse  reports  from  ASR-9  sensors.  A  total 
of  8,588,036  radar  reports  were  analyzed;  4,334,946 
from  ASR-9  sensors  and  4,253,090  from  ASR-11 
sensors. 

1.  Results 

Azimuth  Error  Comparison 

Figures  2  through  4  show  the  estimated  azimuth 
error  distributions  for  the  ASR-9  and  ASR-1 1  surveil¬ 
lance  radars.  Figure  2  shows  the  results  for  the  full 


Figure  2.  Overall  azimuth  error 


Figure  3.  Azimuth  error  for  ranges  <40  NM 


Figure  4.  Azimuth  error  for  ranges  40-60  NM 


Table  I.  ASR-9  Sensors 


Site  Location 

Site  ID 

Baltimore,  MD 

BWI 

Chicago,  IL 

ORD 

Boston,  MA 

BOS 

Los  Angeles,  CA 

LAX 

Manchester,  NH 

MHT 

New  York,  NY 

JFK 

Newark,  NJ 

EWR 

Table  II.  ASR-11  Sensors 


Site  Location 

Site  ID 

Colorado  Springs,  CO 

COS 

Columbia,  MO 

cou 

Lafayette,  LA 

LFT 

Saginaw,  MI 

MBS 

Stockton,  CA 

SCK 

Waco,  TX 

ACT 

West  Palm  Beach,  CA 

PBI 

data  set.  Figure  3  shows  the  azimuth  errors  for  the 
subset  of  reports  40  NM  or  less  from  the  radar 
antenna,  the  ranges  where  3  NM  separation  minima  is 
permitted  for  the  ASR-1 1 .  Figure  4  shows  the  azimuth 
error  for  the  subset  of  reports  40  NM  to  60  NM  from 
the  radar  antenna,  the  extended  range  at  which  3  NM 
separation  minima  is  permitted  for  the  ASR-9  but 
not  for  the  ASR-11.  Each  plot  contains  a  cumulative 
distribution  function  (CDF)  and  probability  density 
function  (PDF)  of  the  errors  for  both  the  ASR-9  and 
the  ASR-11.  The  ASR-9  error  PDF  is  represented 
by  the  light-blue  histogram  and  the  ASR-11  PDF 
is  represented  by  the  red  outlined  histogram.  The 
ASR-9  error  CDF  is  plotted  as  a  blue  line  and  the 
ASR-11  error  CDF  is  plotted  as  a  red  line.  The 
right  y-axis  refers  to  PDF  density  values,  while  the 
left  y-axis  refers  to  the  CDF  density  values.  The 
statistical  standard  deviations  of  the  ASR-9  and  ASR- 
1 1  distributions  are  recorded  in  the  plot  legend. 

Figure  5  plots  the  estimated  azimuth  error  as  a 
function  of  range  and  altitude.  The  top  plot  contains 
the  ASR-9  results,  while  the  bottom  plot  contains  re¬ 
sults  for  the  ASR-11.  The  x-axis  represents  the  range 
of  the  targets  and  the  y-axis  the  altitude  of  the  targets. 
Dotted  lines  representing  elevation  angles  are  drawn 
on  the  plot  for  reference.  The  plots  are  broken  down 
into  cells  0.5  NM  in  range  by  2,000  feet  in  altitude. 
Each  cell  is  assigned  a  color  corresponding  to  the 
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Figure  5.  Azimuth  error  vs.  range  and  altitude 


Figure  6.  Azimuth  error  vs.  elevation  angle 


magnitude  of  the  standard  deviation  of  the  azimuth 
error  of  the  reports  in  the  cell.  The  colormap  ranges 
from  dark  blue,  representing  small  azimuth  errors, 
to  red,  representing  large  azimuth  errors  (standard 
deviation  of  greater  than  0.2  degrees).  The  colored 


cells  are  then  interpolated  to  a  finer  resolution  for 
a  smoother  image.  The  azimuth  error  value  for  each 
color  is  shown  in  the  colorbar  to  the  right  of  the  plots. 
White-space  represents  areas  where  no  target  reports 
were  recorded.  Figure  6  plots  the  estimated  azimuth 
error  standard  deviation  versus  elevation  angle  for 
each  sensor  type. 

Figures  3  and  4  illustrate  that  the  estimated  az¬ 
imuth  error  characteristics  for  the  ASR-11  are  nearly 
identical  to  those  for  the  ASR-9.  Most  notably,  ASR- 
11  azimuth  errors  for  reports  at  ranges  of  40  to  60 
NM  appear  to  be  equivalent  to  the  azimuth  errors 
for  the  ASR-9  at  the  same  ranges.  This  conclusion 
is  reinforced  in  Figure  5,  where  the  right  halves  of 
the  ASR-9  and  ASR-11  plots  display  the  same  error 
levels. 

Figures  5  and  6  show  azimuth  error  increases 
rapidly  with  elevation  angle  for  both  sensor  types. 
This  behavior  is  not  unexpected  as  the  accuracy 
of  monopulse  systems  degrades  at  higher  elevation 
angles  [4],  The  results  are  encouraging  from  an 
analysis  perspective,  as  they  demonstrate  that  the 


analysis  method  captures  an  expected  degradation  in 
performance. 

The  results  show  that  for  reports  within  60  NM 
of  the  radar,  elevation  angle  is  the  dominant  factor  in 
radar  azimuth  accuracy,  while  range  and  altitude  have 
relatively  little  effect.  Figure  3  shows  both  sensors 
having  larger  azimuth  errors  at  short  ranges  compared 
to  long  ranges  (Figure  4),  but  this  behavior  is  expected 
due  to  the  azimuth  error  dependence  on  elevation 
angle.  Aircraft  flying  at  the  same  altitudes  will  be 
at  higher  elevation  angles  relative  to  the  sensors  at 
short  ranges  than  at  long  ranges,  so  subsequently  the 
azimuth  errors  will  be  larger  at  short  ranges. 

Figures  3  and  4  show  that  for  the  data  set 
analyzed,  the  ASR-11  surveillance  radar  is  capable 
of  occasionally  receiving  reports  at  higher  elevation 
angles  than  the  ASR-9  surveillance  radar.  However, 
the  number  of  reports  received  at  high  elevation 
angles  (>  45  deg)  by  the  ASR-11  was  relatively 
small;  roughly  3,500  out  of  4,000,000  reports,  and 
had  minimal  effect  on  the  final  results. 

Conclusion 

The  performance  of  the  ASR-9  and  ASR-11 
surveillance  radar  were  compared  using  a  large  collec¬ 
tion  of  estimated  radar  errors.  The  results  found  that 
the  azimuth  error  of  ASR-11  surveillance  radar  (cr  = 
0.0605  deg)  was  similar  to  that  of  ASR-9  radar  (c  = 
0.0596  deg).  In  particular,  for  targets  in  the  region 
of  interest,  40  to  60  NM  from  the  radar  antenna, 
the  ASR-11  azimuth  error  (a  =  0.0438  deg)  was 
equivalent  to  the  ASR-9  azimuth  error  in  the  same 
region  (cr  =  0.0474  deg).  The  results  of  this  analysis 
support  the  supposition  that  the  40  NM  limit  on  3  NM 
separation  minima  for  ASR-1 1  surveillance  radar  may 
be  unnecessarily  restrictive. 
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